1. Field of the Invention
The present invention relates to a solid state image sensing device in which the image quality of a reproduced image is improved, and a nonlinear resistor circuit suited to an analog image signal processor for use in the device.
2. Description of the Related Art
Recently, since a solid state image sensing device such as a CCD is smaller and lighter than a camera tube, it is often used in image sensing equipment for private use such as a home VCR. In addition, high-definition, low-noise image sensing equipment has been progressed as IC manufacturing techniques have been developed in these days.
However, the conventional solid state image sensing device had such as following problem. In an automatic diaphram the conventional solid state image sensing device, the diaphram mechanism is properly operated to a center pixel on a screen which is identified as a reference luminance. But since the diaphram is not properly operated for a pixel around the reference pixel having a extremely different from reference luminance, the image quality of a reproduced image is deteriorated.
In other hand, image processing techniques and applied techniques of the image processing techniques have been conspicuously developed to be widely applied in the fields of, e.g., a high-quality television, computer graphics, medical service, robot technology, and image recognition.
For example, image processing includes the following fields.
1) A field of reproduction of an original image, in which image transmission is performed and a detected image is reproduced and reconstructed.
2) A field of image conversion, in which a new image is formed from an original image, to emphasize information required by a user.
3) A field of recognition, in which an image further subjected to various processing steps is used to perform recognition at a higher level.
As described above, a range included in accordance with processing contents is very wide.
An image processing system will be described below as an image sensing system. The image sensing system, for example, has an arrangement as shown in FIG. 1.
An image signal output from an image sensing element (senor) 600 using, e.g., a CCD is converted into a digital signal by performing amplification, shaping and noise elimination by a pretreatment unit 610. Thereafter, the image signal is supplied to and processed by an exclusive digital signal processor 620. Since a large amount of data must be processed for the image signal, the pretreatment unit 610 and the digital signal processor 620, both of which can operate at a very high speed, are prepared to sequentially perform processing in units of pixels.
As described above, the conventional image processor is arranged to sequentially process all pixel information by using a small number of high-speed signal processing units. In order to process a larger number of pixels, a processor in which a large number of pixel signal processing units are arranged to process pixel signals in a pipeline manner is developed in consideration of the parallel processing. However, even when pixel signals are to be processed in a pipeline manner, signal processing for each pixel value must be performed at a considerably high speed.
An attempt has been made to execute image processing not sequentially by using a digital processing system but parallel by using an analog circuit network. In this attempt, a signal processing circuit for processing the luminance of a pixel constituted by a image sensing device such as a CCD is arranged for each pixel, and adjacent devices are coupled by a resistor element or a coupling circuit having a function equivalent to that of the resistor element, thereby arranged a circuit network for executing processing. This circuit network is suited to real-time signal processing because it performs arithmetic operations between the pixels completely parallel. An apparatus for executing processing using this circuit network realizes a high speed by the parallel operation of the processing system, and does not essentially depend on the advance in process techniques such as micropatterning of elements. Therefore, the apparatus can be manufactured at low cost.
For example, FIG. 2A shows a state of coupling of a circuit network in which pixels each constituted by a CCD image sensing device are arranged in a matrix form. Pixel value signal processors 100 arranged in a one-to-one correspondence with the pixels are coupled by coupling circuits (resistor elements) 200. FIG. 2B shows coupling of one pixel shown in FIG. 2A with adjacent pixels. A linear resistor, for example, is used as the coupling circuit 200. The function of the circuit network shown in FIGS. 2A and 2B is a Laplacian operation with respect to each pixel value. In other words, this function is of a spatial low-pass filter. Therefore, an output in which RF components such as noise components are reduced is obtained from the circuit network shown in FIGS. 2A and 2B.
FIG. 3A shows a circuit network as a low-pass filter in which pixels are arranged in a triangular form, and FIG. 3B shows coupling of one pixel shown in FIG. 3A with adjacent pixels. Although the circuit network shown in FIGS. 3A and 3B performs the Laplacian operation, it has an advantage of less image direction dependency than the circuit network of the matrix form shown in FIGS. 2A and 2B.
The characteristics of a linear circuit network according to the reference of C. Mead, Analog VLSI and Neural System, Reading, 1989, Addisson Wesly, Appendix C, will be described in detail below by taking a one-dimensional circuit network having an infinite length shown in FIG. 4 as an example in place of the above-mentioned two-dimensional resistor circuit networks.
In FIG. 4, assume that a voltage source {v.sub.k }.sub.k is an input signal and a node potential series {u.sub.k }.sub.k is an output signal. Since the system shown in FIG. 4 is linear, an input/output relationship is represented by an impulse response {h.sub.k }.sub.k as follows: ##EQU1## The impulse response {h.sub.k }.sub.k is calculated from the above equation. Assuming that v.sub.O .noteq.0 and v.sub.k =0 (k.noteq.0), the following equation is obtained by an Ohm's law and a Kirchholff's law: ##EQU2## By eliminating i.sub.k-1 and i.sub.k from equation (1), the following equation (2) is obtained: ##EQU3##
Assume that: EQU u.sub.k =.sub..gamma..sup.k.u.sub.O ( 3)
In this case the following equation (4) is obtained: ##EQU4## Note that only a solution of 0&lt;.gamma.&lt;1 is employed by a boundary condition u.sub.k .fwdarw.0 (k.fwdarw..infin.).
An initial value u.sub.0 of equation (3) will be obtained. FIG. 5 shows an equivalent circuit of the circuit network shown in FIG. 4 obtained when an impulse signals input to k=0. Assuming that an input impedance of the entire network on the right side from a node u.sub.0 (i.e., a network of k&gt;0) is R.sub.in, when both sides of equation (1) for k=0, that is: EQU i.sub.0.R.sub.1 =u.sub.0 -u.sub.1
are divided by u.sub.0, the following equations are obtained: ##EQU5## Therefore, from EQU r.sub.in.sup.-1.R.sub.1 =1-.gamma.,
the following equation (5) is obtained: ##EQU6##
Since the input impedance of the entire circuit network viewed from the input power source v.sub.0 is EQU R.sub.0 +R.sub.in //R.sub.in,
u.sub.0 to be obtained is given by: ##EQU7## where // is the arithmetic operation symbol indicating a resistance obtained when resistors are parallel-connected. When equation (6) is substituted into equation (3), the following equation (7) is obtained: ##EQU8##
Therefore, the impulse response of the circuit network shown in FIG. 4 is obtained as follows: ##EQU9##
That is, the impulse response of the circuit network shown in FIG. 4 has a shape horizontally symmetrically, exponentially decaying from an impulse input node as a vertex.
When the following equation (9) is substituted into equation (8), ##EQU10## h.sub.k is given as follows: EQU h.sub.k =.alpha..gamma..sup..vertline.k.vertline. ( 10)
A case wherein a step input is supplied to the circuit shown in FIG. 4 will be considered.
That is, when ##EQU11## is input, an output is given by: ##EQU12##
FIG. 6 is a graph showing equation (12). As is apparent from FIG. 6, the circuit network shown in FIG. 4 has low-pass filter characteristics. FIGS. 7 and 8 show actual simulation results of a one-dimensional circuit network having 24 nodes, in each of which the abscissa indicates a node number and the ordinate indicates the voltage of a node. FIG. 7 shows an input signal, and FIG. 8 shows an output signal. The input signal is obtained by adding noise to a step signal which is not 0 when k.ltoreq.11 but almost 0 when 12.ltoreq.k.ltoreq.24. #n (n=1, 2, 3, and 4) indicates inputs obtained when an average value of v.sub.s is set to be 0.97, 1.95, 2.90, and 3.85. As shown in FIG. 8, the value of output signal smoothly changes around the step. As is apparent from equation (12), a degree of this "smoothness" is determined by .gamma. and by a resistance ratio R.sub.0 /R.sub.1 from equation (4). Actually, .gamma. is monotonously decreased from 1 to 0 when the R.sub.1 /R.sub.0 is increased. Therefore, the gradient of the curve shown in FIG. 8 is increased by increasing the R.sub.1 /R.sub.0. This tendency is not limited to the one-dimensional circuit network as shown in FIG. 4 but almost similarly occurs in the two-dimensional circuit networks shown in FIGS. 2A to 3B.
When removing noise by an arithmetic operation using the linear circuit network for performing image signal processing parallel as described above, a signal is largely distorted to produce a "blur" in a portion where the signal value largely changes, i.e., a portion near the edge of image information, as shown in FIG. 8. This distortion makes it difficult to simultaneously perform noise removal and edge extraction from image information. Since, however, the above two processing steps are basic elements of the image processing, they are preferably performed at the same time.
In order to solve the above problem, there is proposed the use of a nonlinear resistor circuit in which the resistance nonlinearly changes in accordance with a voltage across the two terminals of the coupling circuit 200 shown in FIGS. 2 and 3 as the coupling circuit 200. The nonlinear resistor circuit basically has a low resistance at a threshold value or less at which a potential difference is present between the two terminals and a high resistance (including infinity) at the threshold value or more. There is the following relationship: ##EQU13##
A function obtained when the nonlinear resistor circuit as described above will be described below by taking the one-dimensional circuit network shown in FIG. 4 as an example. In the one-dimensional resistor circuit network shown in FIG. 4, if resistors R.sub.1 constituting the coupling circuit are linear resistors, each resistor R.sub.1 is kept constant regardless of a potential difference between the two terminals of the resistor. Assuming that coupling obtained by the resistor R.sub.1 is the "strength" of an interaction between nodes, this strength remains unchanged regardless of the state of the circuit network. Therefore, averaging and smoothing of pixel values are performed by this interaction. However, it is not checked whether a difference between two adjacent node signals is caused by noise or an edge. As shown in FIGS. 7 and 8, a step response in the linear resistor circuit network is smoothed by averaging and removing noise superposed on a step signal and averaging an edge portion at the same time.
The above nonlinear resistor circuit is applied as each resistor R.sub.1 of the resistor circuit network as shown in FIG. 4. Whether a difference in node values between couplings is caused by discontinuity due to an edge or by noise is checked in accordance with a potential difference between the two terminals of the nonlinear resistor circuit, thereby determining whether averaging is to be performed. That is, if the potential difference between the two terminals of the coupling circuit is a predetermined threshold value or more, it is determined that the discontinuity is caused by an edge, and no averaging is performed. If the potential difference between the two terminals of the coupling circuit is the predetermined threshold value or more, it is determined that the discontinuity is caused by noise, and averaging is performed. In other words, the position of an edge is detected in accordance with whether the potential difference between the two terminals of the coupling circuit exceeds the threshold value, the circuit network is divided or formed into an independent network at the edge portion. If the potential difference between the two terminals of the coupling circuit exceeds the threshold value, averaging is not performed for the edge portion but performed in each independent network. Even when the circuit network is extended to be two-dimensional, the circuit network is disconnected with respect to a node potential change large enough to determine an edge, and no smoothing is performed, as in the one-dimensional circuit network.
The related arts cited in the above description are disclosed in the following references.
1) Wataru Miyao, Masaya Yabe ed., "Intellectualization Sensing", (Science Forum K. K.), pp. 169-185
2) J. Hutchinson et al., "Computing Motion Using Analog and Binary Resistive Networks", IEEE COMPUTER, Mar. 1988, pp. 52-63
3) H. Kobayashi et al., "An Active Resistor Network for Gaussian Filtering of Image", IEEE J. Solid State Circuits, Vol. 25, No. 5, May 1991, pp. 738-748
4) C. Mead and M. Mahowald, "A Silicon Model of Early Visual Processing", Neural Networks, Vol. 1 (1988), pp. 91-97
5) J. Harris, C. Koch, J. Luo and J. Wyatt, "Resistive Fuses: Analog Hardware for Detecting Discontinuities in Early Vision"; Analog VLSI implementation of neural system, 1989, Kluwer Academic Publishers
6) H. Hayashi et al., "An Analog CMOS Network for Gaussian Convolution with Embedded Image Sensing", 1990, IEEE International Solid-State Circuit Conference, pp. 216 (1990)
7) D. L. Standley et al., "An Object Position and Orientation IC with Embedded Imager", 1991 IEEE International Solid-State Circuit Conference, pp. 38 (1991)
8) C. Mead: "Analog VLSI and Neural Systems", Reading, Addisson Westey, 1989, APPENDIX C.